IIGIIillllllllilllMlililillillllll 

US006292909B1 

(12) United States Patent (lo) Patent No.: us 6,292,909 Bi 

Hare (45) Date of Patent: Sep. 18, 2001 



(54) APPARATUS FOR TESTING 

COMMUNICATION EQUIPMENT 

(76) Inventor: Duncan Hare, 9782 Garrett Cir., 
Huntington Beach, OA (US) 92646 

( * ) Notice: Subject to any disclaimer, the term of this 
patent is extended or adjusted under 35 
U.S.C. 154(b) by 0 days. 

(21) Appl. No.: 09/115,066 

(22) Filed: Jul. 14, 1998 

Related U.S. Application Data 

(60) Provisional application No. 60/052,430, filed on Jul. 14, 
1997. 

(51) Int. Cl.^ H02H 8/05 

(52) U.S. CI 714/40; 379/10 

(58) Field of Search 714/40; 379/29, 

379/10, 88.18; 370/242; 455/426, 427 

(56) References Cited 

U.S. PATENT DOCUMENTS 

5,572,570 11/1996 Kuenzig 379/1 

5,737,518 * 4/1998 Grover et al 714/38 

5,835,565 11/1998 Smith et al 379/5 

6,052,371 ♦ 4/2000 I^mieux 370/395 



OTHER PUBLICAHONS 

HP Openview Network Node Manager for the Windows 
NT® Operating System Product Brief, Hewlett Packard, 
International Data Corporation, Mar. 1996, pgs 1-6. 
itSMF Newsletter Issue No. 32— Aug. 1998. 

* cited by examiner 

Primary Examiner — Robert Beausoleil 

Assistant Examiner — Rita A. Ziemer 

(74) Attorney, Agent, or Firw— Howison, Chauza, Thoma, 

Handley & Amott, L.L.P.; Roger N. Chauza, Esq. 



(57) 



ABSTRACT 



Apparatus for monitoring and testing a transaction system, 
and controlled by enterprise automation equipment. The 
testing apparatus is programmed with various MIB struc- 
tures to provide probe points and signals to the transaction 
system. Responses are received by the test apparatus via 
check points connected to the transaction system. The test 
apparatus applies test signals to the probe points in a manner 
similar to signals applied by users of the transaction system. 
The responses received from the transaction system are 
again similar to the responses received by the user in the 
normal utilization of the transaction system. 

5 Claims, 5 Drawing Sheets 
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APPARATUS FOR TESTING nected to the various subsystems and componenls of the 

COMMUNICATION EQUIPMENT system. In addition, the traffic driver/checker includes check 

points connected to various other equipment of the system to 

RELATED APPLICATION be monitored and tested to obtain inputs in response to tests 

^. , ^ n r ' „ j5 carried out. The traffic driver/checker is programmed with a 

-nijs apphcauon claims the benefit of prior-flled <^ Management Information Base (MIB) that controls the flow 

'J ^no. '^PP^i^^'i"" No. 60/052,430 filed Jul. ^„ ^^^^ transaction system. The traffic driver/checker 

14, 1997, the disclosure of which is incoojorated herein. ^^^^^^^ conjunction with a conventional enterprise auto- 

TECHNICAL FIELD OF THE INVENTION mation package to perform the surveillance, monitoring and 

10 testing of the transaction system. 

The present invention relates in general to testing 

apparatus, and more particular to apparatus and methods for BRIEF DESCRIPTION OF THE DRAWINGS 

testing the operability of mction systems to find errors p^^her features and advantages will become apparent 

and faults, as experienced by customers and users of the ^^^^ following and more particular description of the 

systems. 15 preferred embodiment of the invention, as illustrated in the 

BACKGROUND OF THE INVENTION accompanying drawings in which like reference characters 

generally refer to the same parts, elements or functions 

The networking of computers and the computerized throughout the views, and in which: 

equipment has lead to substantial enhancements in the piG, 1 is a block diagram illustrating the environment in 

accessibility and distribution of data and information. The which the invention can be advantageously practiced; 

networking of equipment can be accomplished by local area j is the management information base (MIB) struc- 

networks, wide area networks and other types of commu- ^^^^^ according to the preferred embodiment of the inven- 
nication systems, such as the public switched telephone 

network (PSTN). While the integration of geographically ^* , . i mi . r 

remote equipment is substantially facUitated by networks of ^5 FIG. 3 is a user screen d^play illustrating the number of 

areas selectable for use; 



various types, the failures in such networks can and systems 



have substantial and wide-spread ramifications. Indeed, the FIG- 4 is a user screen display iUustrating a promotion 

troubleshooting of networked systems becomes more window; 

troublesome and complicated. There exist test operations FIG. 5 is a user screen display showing a test case editor; 

and methods that can carry out periodic tests on a network FIG. 6 is a user screen display illustrating a lest executive 

system and report failures. In many instances, the test window; 

circuits are driven by software embedded in the application j iUustrates a screen display showing a test list with 

program. While this may be effective, a failure of the corresponding test results; and 

application system often results in the inability to report ^ ^ ^^^^^^^ ^ interrelationship 

failures. As a result the customers or users are the first to ^^^^^^ ^^.^.^^^^ ^^^^ 

expcnence moperabihty of the system, rather than a repair cases 
or technical staff. 

From the foregoing, it can be seen that a need exists for DETAILED DESCRIPTION OF THE 

a technique to carry out test on transaction equipment in a INVENTION 

manner substantially the same as the procedures utilized by pj^ ^ iu^strates a block diagram of the environment in 

a customer or user. Another need exists for a procedure to ^^.^^ preferred embodiment of the invention is adapted 

test the equipment in the manner utihzed by customer so operation. The invention is weU adapted for use with 

that an accurate portray of the fault can be experienced by transaction systems, such as automatic call distributors 

the test operation. Another need exists for a test technique ^^^^^y Conventional ACD equipment is accessed through 

for finding faults m the equipment before customers, thereby ^^^^ ^^^^^^^^ telephone network (PSTO) 20. The 

reducing customer dissatisfaction with the system. Another p^^^ ^0 is coupled to an interactive voice response ACD 

need exists for a test method for carrying out tests on ^2 by way of telephone lines. The I VR ACD 22 is, in 

systems at periodic time intervals to verify the operabihty of ^^^^ ^^^^^^^ ^^^^ interactive voice response units 

the system. ^^^^ contain the IVR call flow sequences, one shown as 

SUMMARY OF THE INVENTION reference 24. The IVR call flow retrieves data from legacy 

systems, associates data to the individual calls, and transfers 

In accordance with the principles and concepts of the calls to agent ACD queues. The IVR units 24 can be 

invention, the shortcomings and disadvantages of the prior conventional Unix systems programmed to provide various 
art systems have been overcome. According to the preferred 55 types of voice responses. The IVR units 24 are coupled to an 

embodiment of the invention, the test system accesses a agent ACD system 26, The agent ACD system 26 is coupled 

transaction-type system in a manner much like the users to and delivers queued calls to plural agent telephone sets, 

thereof. The responses of the transaction system are much one shown as reference 28. As is common, an agent or 

like those received by the users of the system. Based on the operator working with the telephone set 28 also has avail- 
test conducted, and the response thereto, it can be deter- go able a terminal or other computer-type equipment 30. As the 

mined if the system is functional, and if not, which sub- agentisdiscussingwiththecaUingparty on the telephone set 

system is at fault, llie invention is integrated with conven- 28, the terminal 30 can be utilized to call up data related to 

tional test systems so that the test can be carried out the calling customer. A database concerning matters related 

periodically, and malfunctions reported quickly to the appro- to the various calling customers is stored in a mainframe 
priate technical staff or repair persons. ^5 computer 32. ITie mainframe computer is coupled to a 

In accordance with an important feature of the invention, computer telephony integration (CU) unit 36 via a data 

a traffic driver/checker is provided with probe points con- gateway 34. The CTI provides screen displays or pops to the 
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terminals 30. The CTI unit 36 is coupled to an application ACD system, in the nature of check points 48 are obtained 

server 38 which, in turn, is coupled to the agent terminal 30. from the computer telephony integration unit 36, the agent 

The overall ACD system functions to receive incoming terminal 30 and the agent telephone unit 28. Two mecha- 

calLs and distribute the calls to the various agent positions nisms arc utilized for system surveillance of the ACD 

that are not then busy. The agents can then verbally discuss 5 system. First, polling the system (especially the individual 

matters with the calling customer, call up data concerning ivrs) and, second, monitoring customer trafiBc. The first 

the customer, and provide human assistance. As can be mechanism verifies the correct operation of the I VRs and the 

appreciated, the monitoring and testing of the system for cTI components of the system. The second monitor skills 

operability is important so that any faults are detected early requested by the customers, and statistically verifies that 

and repaired such that the system functions optimally. While ^^Hg ^re routed to the correct agents for the skills requested, 

the preferred embodiment is described in conjunction with openview equipment 42 continuaUy polls the system 

an ACD system, other transaction systems, such as legacy through the traffic driver/checker 44. Both the PSTN 20 

systems, and other systems, are applicable to use by the directory numbers and the IVR individual numbers are 

invention. polled. When a failure is detected the operations center 40 

In order to provide surveillance and monitoring for the ^5 probes the system to determine which component failed. A 

ACD system, the invention employs conventional equip- failure can be a situation when a call is not completed 

ment adapted for managing local area networks. In the through the system to a virtual agent. As noted above, when 

preferred embodiment, utilized in conjunction with the the failed component is identified by the rules programmed 

invention is an enterprise automatic package, such as the in the operations center 40, a trouble ticket and alert notice 

Hewlett-Packard IT operations center equipment 40. The 20 are dispatched to the appropriate groups for tracking and 

operations center 40 is programmed therein the rules, poh- repair. 

cies and instructions for generating trouble tickets concern- preferred embodiment, the traffic driver/checker 44 

ing any malfunctions of the system to be monitored. In -^^j^^^^ ^ ^^^^^j computer with a Pentium 166 MHz 

addition, the operations center 40 functions in conjunction processor, 128 MB of RAM, 15 megabytes of hard disk 

with Hewlett-Packard openview equipment, identified by 25 drive, Windows NT 4.0, a Dialogic Tl board, and other line 

reference 42. m openview equipment 42 includes a system interface cards or circuits. 

map for providing SNMP control and a polling rate for ^ .„ , ,,tt^ . 1 -.l- 

car in out tests ^ <=> pjQ 2 illustrates the MIB structure programmed within 

carrying ou es . - . the traffic driver/checker 44. The MIB structure 50 includes 

In accordance with an important feature of the invention, ^ j. „^,/„u^„i, \aiu o^t«,o t^ot kaiu ^ t^ct 

•J J • . re J • / i_ 1 ><j *L . • 1 J 1 1 a tramc driver/check MIB 52, an active test MIB 54, a test 

provided IS a traffic dnverMecker 44 hat includes plural 30 individual test case MIB 58. The 

output checkpoints 48 coupled to the system to bemonitored ^^^^ driver/checker MIB 52 describes the traffic driver/ 

and tested. The driver/checker 44 includes an SNMP proxy ^^^^^^^ ^^.^ j^^j ^ , ^^^^^^^^ 44 

agent that operates under the con rol of the open view ^^^^ ^^^.^^ ^^^^^ driver/checker MIB 52 may 

equipment 42, through a system MIB. Th, traffic driver ^^^^f^ ^^^^ y^^^g^^ (qjo) of the active test set MIB 

checker 44 can communicate With the Openview equipment 35 e>i u * *■ *r • ^ Tn,^ c^t njim 

. r t * .1 jj-.- *u * J- / 54 when a test is utihzing the port. The active test set MIB 

42 by way of the Internet. In addition, the traffic driver/ j ^ m, a ««„*,^iro^t;,fl t»cfc ^o™^ toct tu^ 

, c . \ u ' . A£ 54 describes and controls active tests carried out to test the 

checker 44 includes a number or output probe points 46 .^^^ * t-u- kait* • n a u *u cmajtd • 

^ r ^ . ACD system. This MIB is polled by the SNMP manager in 

connected to the vanous sub-systems of the ACD systertL ^^^^ ^^^^ ^j^.^^ ^^^^^^ ^^^^ ^ ^^^^^ 

the open v,ew equipment 4^ continually polls t^e AUU ^^^^ ^^.^^^^ ^^^^ 

cases to run as a 

system through the traffic dnver/checker 44. Both the direc- 40 u .u cktajth „ « a 

/ . f .u ncTTwT -lA A T^m ' A' -A 1 Tesult of thc SNMP manager "get command, 

tory numbers of the PSTN 20 and the IVR individual ? , .,.11 

numbers are polled. When a failure is detected, the opera- Th^ ^^^t case MIBs 56 describe individual test cases, 

tions center 40 probes the system, via the traffic driver/ ^IBs are defined so as to be extensible by the user, 

checker 44, to determine which component or sub-system either by addmg new test case MIBs m associated action 

failed. When the failed component is identified by the rules 45 ^^^^^ ^"^'^"^ genenc test case MIB 

programmed in the operations center 40, a trouble ticket ^^at executes any user provided windows MT program as a 
together with an alert is sent to the appropriate office for 

tracking and repair. The open view equipment 42 provides in The test case MIBs 56 have two sections, namely the test 

a standard manner SNMP mechanisms to control the traffic setup and test execution sections. In a set of test cases, all 

driver/checker 44. The open view equipment 42 further 50 test set up sections of the MIBs are executed before the test 

provides polling control and obtains and sets variables in the execution sections. The test set up sections of the test case 

driver/checker 44. The operations center 40 contains the MIB is synchronous, meaning that it does not return a 

policies and control to utilize the probe points 46 provided "success" value to the "SNMP get" command until the test 

by the driver/checker 44 to complete surveillance and deter- set up is complete. 

mine the failing component when outages are detected. 55 The openview equipment 42 communications with the 

In the preferred form of the invention, the probe points 46 traffic driver/checker 44 by using a system MIB. The system 

provided by the traffic driver/checker extend as directory MIB is shown in Tables 1-6 below. The OID of the elements 

numbers to the PSTN 20, as queue directory numbers to the in the MIB is prefixed with the notation 1.3.6. 1.4.1. [999], 

interactive voice response ACD unit 22, as test ports and where the following relationship exists: 

respective directory numbers to the individual voice 60 1 equals ISO 

response units 24, as queue directory numbers to the agent 3 equals ORG 

ACD unit 26, as transactions to the mainframe computer 32 ^ equals DOD 

and as a test program on each IVR to the data gateway 34. ^ e uals Internet 

Various data or information can be input by the traffic ^ 

driver/checker 44 to the ACD system to determine operabil- 65 ^^^^^ pnvate 

ity thereof. In the case of the PSTN 20, a telephone number [999] equals the business controlling the testing apparatus 

can be dialed to access the ACD system. Responses from the 1 equals the traffic driver/checker MIB 44. 
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The traffic driver/checker static station of the M IB is set 
forth below. The OID for each MIB element must be unique. 
The "n" in the OID enables multiple agents. For example, n 
equal 1 for the first agent, n equal 2 for the second agent, and 
so on. 

TABLE 1 



MID variaDie 


tJDject lu 


'lYpc of 
data 


iNumoer in mid 


TrafiBc 


.[99911.1 


List 


One 


driver/checker info 








Port Information, 


.(999J1.1.1 


Number 


One 


number of entries 








Port Information 


.[999].1.1.1.D 


List 


Multiple Ports 


Port Identifier 


.[9991 1.1. l.D.l 


Card 


One Per port 






(Trunk) 








Number, 








Charmel 








Number 




Port phone 


.[999ll.l.l.n.2 


Phone 


Used to match calling 


number 




Number 


event with ringing 








event. 


Agent Information 


.[999ll.l.2.n 


List 


One Agent Per Port 


Agent DN 


.[99911.1. 2.n.l 


Phone 


One per agent 






Number 




Agent Login 


.[9991 1.1. 2.n.2 


Phone 


One per agent 






Number 




Active Test Set 


.[999ll.l.2.n.3 


OID 


OID of test running 








on port. 



TABLE 2 



TABLE 3 



Number 



^ MIB Variable 


Object ID 


T^e of data 


in MIB 


Test case list, number 


.[99911.2 


Number 


One 


of test case entries 








Test case list, 


.[99911.2.3.n 


Number 


One per test 


number of test 






list entry 


cases in list 








10 OID of test 


.[999ll.2.3.n.m 


OID 


OID list of 


case. 






test run to 








get result. 



The traffic driver/checker test set or dynamic section of 
the MIB is as follows. In this section of the MIB, the "n" in 
the OID enables the traffic driver/checker to execute mul- 
tiple active tests. For example, n equal 1 for the first active 
test, n equal 2 for the second active test, and so on. 



15 



20 



25 



30 



35 



The test case section of the MIB is as follows. In this 
section of the MIB, the "n" in the OID gives each test a 
unique OID. For example, n equal 1 for the first test case in 
a section, n equal 2 for the second test case in a section, and 
so on. 

The generic test case section of the MIB is illustrated 
below. This shows the test case set up, test case execution 
structure to the test case. 







Type of 


40 


MIB Variable 


Object ID 


data 


Number in MIB 


Active Test Set List 


.[99911.3 


Number 


Number of test cases 


Active Test Set 


.[999ll.3.n 


OID 


OID of Test case list 
for this active test set. 


Test Set Status 


.[999ll.3.n.l 


Enumer- 
ation 


Active 45 

Inactive 

Complete 

Broken 


Test Set Resource 


.[99911.3.n.2 


Enumer- 


Running 


Status 




ation 


Pass 




.[999ll.3.n.3 




Fail 5Q 


Current Active 


OID 


Running-OID of 


Test Case 






cunent test case 
Pass-OID of test 
case List 
Fail-OID of first 
test that failed in test 
case list. 



In this section of the MIB, the "n" in the OID enables the 
traffic driver/checker 44 to contain an unlimited number of 
sequences of test cases. For example, n equal 1 for the first 
sequence of test cases, n equal 2 for the second sequence of 
test cases, and so on. 

In this section of the MIB, the "m" in the OID enables the 
traffic driver/checker to execute a sequence of test to com- 
plete an active test. For example, m equal 1 for the first test 
ca.se, m equal 2 for the second test case, and so on. 



TABLE 4 



MIB Variable 



OID 



Type of 



Number in MIB 



Generic Test Cases 
Test Case List 
Test Case Name 
Test Case Set Up 

Test Case Set 
up 

Test Case 
Execution 
Test Case 
Execution 



.[99911.4 
.[999]. 1.4.1 
.[999]. 1. 4.1. n 
.[999].1.4.1.n.l 

.[99911. 4.1. n.l.m 

.[99911.4.2.n.2 

.[99911.4.1.n.2.m 



Number 

Name 

Name 

String 

Name 

String 



One 

One per test case 
One List Per Test 
Case 

Multiple per test case 

One List Per Test 
Case 

Multiple per test case 



For the test case set up and execution, the "string" is a 
command line, with input fields, that executes a windows 
MT program. 

The voice test case section of the MIB is as follows. 
Expected DNs called are: 

a) call sender, 

b) input ACD queue, 

c) individual IVR, and 

d) agent ACD queue. 

Table 5 below illustrates some generic voice test cases. 



TABLE 5 



MIB Variable 



OID 



Type of data Number in MIB 



60 



65 



Voice Test cases 
Test Case Type 
DN Call List 
Test Case Name 
Test Case Set Up 
Expect IVR 
Greeting 
Test Case 
Execution 
Call Center DN 
Expected 
Average Speed 
of Answer 
(Input) 
Expected 
Variance Speed 
of Answer 
(Input) 



.[999].1.5 
.[999].1.5.1 

.[999].1.5.1.n 

.[999].1.5.1.n.l 

.[999].1.5.1.n.l.l 

.[999].1.5.2.n.2 

.[999].1.5.1.n.2.1 
.[999].1.5.1.n.2.3 



Number 
String 

Vbicc Prompt 



One 

One per test case 
One per test case 
One per test case 

One per test case 



Phone Number One per test case 
Tune (seconds) One per test case 



.[999].1.5.1.n.2.4 Hme (seconds) One per test case 
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TABLE 5-continued 



MIB Variable 


OID 


Type of data 


Number in MIB 


Actual Average 


.[999].1.5.1.n.2.5 


Time (seconds) 


One per test case 


(Measured)r 








Actual Variance 


.[999].1.5.1.n.2.6 


Time (seconds) 


One per test case 


(Measured) 








Test Case 


.[999].1.5.2 


Number 


One 


IVR Response 








List 








Test Case Name 


.[999].1.5.2.n 


Name 


One per test case 


Test Case Set Up 


.[999].1.5.2.n.l 




One per test case 


Expect rVR 


.[999].1.5.2.n.l 


Voice Prompt 


One per test case 


Prompt 








Test Case 


.[999].1.5.2.n.2 




One per test case 


Execution 








IVR Input 


.[999].1.5.2.n.2.] 


String of Digits 


One per test case 


String 









10 



IS 



Table 6 shows an MIB for an agent call. This MIB 
provides information to test a case where an IVR input string 
results in a call to an agent and some data function, such as 
a screen pop, in addition to the call. 



TABLE 6 



Test Case 
IVR Transfer 


.[999]. 1.5 .3 


Number 


One 


Test Case Name 


.[999].].5.3.n 


Name 


One per test case 


Test Case Set Up 


.[999j.l.5.3.n.l 




One per test case 


Expect Agent 


.[999].1.5.2.n.l.l 


Answer 


One per test case 


Call 








Expected Data 


.[999].1.5.2.n.l.2 


Enumeration 


One per test case 


Type 




l-None 








2-Gcncsys 








3-URL 








4-TBD 




Expected Data 


.[999].1.5.2.n.l.3 


String 


One per test case 


Test Case 


.[999].1.5.3.n.2 




One per test case 


Execution 








IVR Input 


.[999]. 1.5.3.0.2.1 


String of Digits One per test case 


String 









TABLE 7 



MIB Variable 


OID 


Type of data 


Number in MIB 


Transaction Tbst 


.[999J.1.6 


Name 


One 


Case 








Data Transaction 


.[999J.1.6.1 


Number 


One 


List 








Data Transaction 


.[999].1.6.n.l 


Name 


One Per Transaction 


Name 








Data Transaction 


.[999].1.6.n.2 


1 RPC 


One Per Data 


Type 




2 LU6.2 
Transaction 

3 HTTP 
(URL) read 


Returned list entry 


Transaction 


.[999].1.6.n.3 


String 


One 


Tbrget 




Target Host/ 




Destination 




Machine 




Transaction Input 


.[999].1.6.n.4 


String 


One 


String 








Data Returned 


.[999].1.6.n.5 


Number 


Multiple 


Number Output 








Strings 








Data Returned 


.[999].1.6.n.m.l 


Number 


One Per Output 


Output String 






String 


Position 








Data Returned 


.[999].1.6.n.m.2 


String 


One Per Output 


Output String 






String 



30 



35 



The transaction section of the MIB is set forth below in 
Table 7. 



45 



55 



60 



65 



For a more thorough understanding of the utilization of 
SNMP and MIBs, reference is made to the text TOTAL 
SNMP: "Exploring The Simple Network Management 
Protocol", by Sean J. Hamedy, Prentice Hall 1997, ISBN 
0-13-6469949; and "Understanding MIBs", by David 
Perkins, Evan McGinnis, Prentice Hall, ISBN 0-13-437708- 
7, the disclosures of which are incorporated herein by 
reference. 

To ensure that customers of the ACD system are satisfied 
with the service level they are receiving, the monitoring 
system tests the services in the same way that a customer 
will use the service. 

The ACD system is monitored in a 6-step process: 

1) Define exactly what services are to be to monitored, 

2) List all the components, both hardware and software 
that are needed to provide the services, 

3) List the number of ICONs needed to represent the 
components on the Graphical Display of the service, 

4) Define the points and the components that should be 
tested, 

5) Define the tests that should be run in order to verify that 
the components are working properly, and 

6) Write the test routines using MIBs and test cases which 
the MIBs will use to verify that the systems are working 
properly. 

These steps are discussed in more detail below. 

The first part of the monitoring procedure is a planning 
process. A list is made of the services that are to be 
monitored. It may be desirable to monitor only a subset of 
the overall service package, or all the services can be 
monitored. 

Once the services to be monitored have been listed, it is 
necessary to identify the steps a customer performs to use 
the service (dial a certain telephone number, enter password, 
identification, et cetera). A definition is also needed of what 
constitutes successful service delivery — who is the 
customer, and what service is expected. 

Next, the various subsystems used in providing each 
service are identified. Systems interdependencies should be 
identified as well. Each node should be identified, along with 
the test cases and each test case*s parameters, plus details as 
to what constitutes a successful test. 

By identifying the services and the various system ele- 
ments that are necessary to provide each service, informa- 
tion is compiled to simulate a customer transaction. What 
constitutes a passing indication for each subsystem and for 
the overall service transaction is also identified. 

For example, consider a service which provides custom- 
ers with voice mail when the access number is dialed. To test 
this service, it is necessary to dial through the telephone 
network to the voice mail system, and attempt to retrieve a 
voice mail message. The information required to write the 
test procedure includes the telephone number to dial, a 
method of confirming that the telephone was answered, the 
voice mail mailbox number, the password, and a message 
stored at the test mailbox. The test case list would include 
two tests: (1) dialing into the system and (2) message 
retrieval. If either test fails, subsequent tests are needed. Did 
the dialup portion of the test fail? Why? Some failures may 
be line busy, no dial tone, or ringing existed but the distant 
end failed to answer the call. 

Additional tests will be needed to discover why voice 
mail retrieval failed. Did the system accept the password? If 
the voice mail system accepted the password, did it fail only 
once (perhaps due to a bad connection) or did it fail 
repeatedly (possibly the voice mail program failed, or was 
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the test equipment unable to access the mailbox because the screen 30 will pass the test. However, all systems from the 

access database table was changed. application server 38 to the mainframe 32 will fail the tests, 

A passing indication is obtained when the test equipment because the test requests cannot pass through the failed 

dials into the system and successfully retrieves the test voice application on the server 38. The test equipment in conjunc- 

mail message from the voice mail system. 5 lion with an enterprise automation package will identify the 

The probe points 46 may be placed at subsystem bound- failed component and facilitate the repair process by elimi- 

ary pointsr-for example, a gateway, a network access point, ^a^ng guesswork in identifying the point of faUure. 

or a network transport system. Probe points may also be Qnce a service assurance test sequence shows a system 

placed within an application system as the transactions or component has failed, the test equipment in conjunction 

remote procedure calls (RPCs). , , with the enterprise automation equipment begins a series of 

Boundary probe points define the subsystena which has diagnostic tests to determine exactly where the failure has 

failed. If tests to a particular probe point fail, but network ^ -n, a- *• i ■ . ^ ,u*u 

. . . ,i_ . • . u L / u 1- occurred. The diagnostic rules associated with the enterprise 

subsystems up to that point have been proven to be working, . . ^ , - r .i. . i_i 

the problem is likely lo be found in the subsystem which automation equipment determine the source of the trouble, 

failed the test ^ 5 As an example, suppose that tests have just been com- 

Intemal probe points define the part of an application or Plet^d on the application server 38, and the CTI server 36 

subsystem which has failed. These probe points are useful ^^s passed its tests. While tests are bemg performed on the 

after a test lo a boundary probe point has failed, because they CTI server 36, the application server fails— hence, the CTI 

provide additional diagnostic information. As one example, tests report a failure. The diagnostic rules then run a series 

if boundary probe points show a mainframe application 20 of diagnostic tests to determine the source of the failure, and 

gateway is not returning requested information, running a optionally route a trouble ticket to the appropriate operations 

test to an internal probe point could help determine that the group. 

problem is not the gateway itself, but rather one or more As noted above, the trafiBc driver/checker 44 is designed 

application end points which are failing to report data to work with an SNMP network management system, such 

received. 25 as HP Open View 40, After a fault condition has been 

Service assurance tests can verify that the systems are identified, the enterprise automation package has the option 

delivering the services expected by the customers. Defining to follow a predefined rule to issue a trouble ticket to the 

these tests requires use of the information already operations group best able to resolve the problem. When the 

gathered — in particular, the definition of what constitutes a trouble has been resolved, the technical personnel can clear 

successful customer transaction, and what a customer needs 30 the ticket. By enforcing a policy requiring the technical staff 

to do in order to obtain the service. to clear the ticket with operations personnel, a complete 

Diagnostic tests are required after a transaction failure has record of the trouble and its resolution becomes available at 

been detected. Using the information identified above, each the management level. 

boundary point is tested to determine which subsystem has The MIB (Management Information Base) is a primary 

failed. If the system is tested in a sequential manner, from 35 component of the monitoring system. The MIB allows one 

beginning to end, the first boundary point which fails the to define precisely what tests that are to be run, in which 

diagnostic test must contain the faulty subsystem. order, and what response is expected from the test. The test 

Once the fauUy subsystem has been identified, running equipment will execute the MIB step by step, and will return 

tests to various sub-elemenLs within it will identify which a pass or a fail condition to the monitoring system, such as 

parts or components have failed. 40 HP OpenView 40. 

The diagnostic rules are applied at the enterprise automa- The MIB structure is essentially a collection of linked lists 

tion level. For example, if a network data path has failed, it and databases, which provide information about the tests to 

is likely there will be several other tests which fail. The be run, the order in which they are to run, and the circum- 

function of the enterprise automation layer is analyze the stances under which they are to run. The first MIB identifies 

results of the test, and isolate the most likely causes of the 45 the test case fists according to ASNl numeric names by the 

failure. module name, otherwise also known as object identifiers 

Consider the example in FIG. 1, where an application on (OID). Table 8 below shows an assortment of different 

the server 38 has failed. If a sequence of tests is initiated via • modules which are contained in the MIB, including several 

the PSTN 20, dialing into the IVR 24, all subsystems up to different test cases, lists of test cases to be run, as well as 

and including the agent position telephone 28 and data some management modules. 



TABLES 



m/b Table O/D Name Status MaxAcccss 



] 


1, 3, 6, 1, 4, 1, 2392, 2, 1 


sigGlobalRegModule 


not- accessible 


2 


1, 3, 6, 1, 4, 1, 2392, 2, 2 


s igSystcmMibCapModu le 


not- accessible 


3 


1, 3, 6, 1, 4, 1, 2392, 2, 3 


systemlntegrationGroupVoicePorth 


not-accessible 


4 


1, 3, 6, ], 4, 1, 2392, 2, 5 


testCascLists 


not-accessible 


5 


1, 3, 6, 1, 4, 1, 2392, 2, 6 


voiccCaintstCases 


not-accessible 


6 


1, 3, 6, 3, 4, 1, 2392, 2, 7 


userTestCases 


not-accessible 


7 


1, 3, 6, 1, 4, 1, 2392, 2,8 


voiceAnswerlts tCases 


not-accessible 


8 


1, 3, 6, 1. 4, 1,2392, 2,9 


AnswcrTestCascs 


not-acccssible 


9 


1, 3, 6, 1, 4, 1, 2392, 2, 10 


TestCases 


not-accessible 


10 


1, 3, 6, 1, 4, 1, 2392, 2, 11 


MCallFIossTestCases 


not-acccssibte 


11 


1, 3, 6, 3, 4, 1, 2392, 2, 12 


MPoitTestCases 


not-accessible 


12 


1, 3, 6, 1, 4, 1, 2392, 2, 14 


s ystem in tegra lio nGroup PBXPor tMI 


not-accessible 


13 


1, 3, 6, 1, 4, 1, 2392, 3, 4 


testCascLists Coot 


not-accessible 
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TABLE 8-coDtinued 



m/b Table O/D 


Name 


Status 


MaxAccess 


14 


1, 3, 6, 1, 4, 1, 2392, 3, 4, 1 


testCase Us tsGroups 




not-accessible 


15 


1,3. 6, 1,4, 1,2392, 3, 4, 1, 1 


nodcNameGroup 


current 


not-accessible 


16 


1,3, 6, 1,4, 1,2392, 3, 4, 1,2 


testCase Us tsGroup 


current 


not-accessible 


17 


1,3, 6,1,4,1,2392,3, 4, 1,3 


tcstCaseUslGroup 


current 


not-accessible 


18 


1, 3, 6, 1, 4, 1, 2392, 3, 4, 1, 99 tcstCaseManageiOroup 


current 


not- accessible 


19 


1, 3, 6, 1,4, 1, 2392, 3, 4,2 


testCaseUstsCo mp is 




not-accessible 



The test equipment provides a "User Test Case table" 
which is integrated into the MIB structure. Using this 
user-defined test case table, external programs can be 
defined and run under conditions set by the user. 

The traffic driver/checker 44 has four sets of tables visible 
to the user. 

The four sets of tables are: 

1. The Node Table 

This contains a list of the node is the system, examples of 20 
components are: the PSTN, IVR pool. Call Center Agents in 
a location, and Mainfirame Transactions. 

2. The Test Case Lists Table 

This is a list (sequence) of test cases. Each test case in the 
list is to be run before the complete test list is passed. A 25 
sequence of test cases will stop on the first failure, or if a test 
case does not respond within the specified timeout. 

A timeout of zero means the test case has no timeout. This 
is used for long running tests (for example and IVR port 
monitor), that may report a failure at a later point in time. 3Q 

3. The Test Case List Table 

This contains the run time information for test cases for a 
specific test list. ITiis includes test case result (pass, fail, or 
timeout), the test case result (a text string), and the test result 
information fi'om all previously successful test cases in the 35 
test case list. 

4. Tables of test cases with similar parameter syntax, but 
different parameter values. 

These are the tables of test cases. These contain the static 
information used to run a test. Static information includes, 
telephone number to dial, program to run, and timeout for 
the test. 

Various user-selectable options are available by clicking 
on areas of the user screen interface of the traffic driver/ 
checker 44, as follows: 45 
Database: 
Initialize 
Expunge 
Promote 

Exit 50 
Testing: 

Components 

Test Cases 
Help: 

File — Load 55 
Testing — Nodes 
Testing — Test Cases 
Test Case Running 
About Service Meter 
"Initialize" is used to initialize the test system. 60 
"Expunge" deletes system component entries which have 
been previously marked for deletion. 

"Promote" moves the contents of the database from the 
"test" test apparatus to the production systems, 
llie "Exit" option terminates the test program. 65 
At the "Testing" menu, one may select either the "Com- 
ponents" section or the "Test Cases" option. 



When the program is first started, the only choice under 
either option is "Load." This loads any existing database 
entries for either the system component list or the test case 
list. 

Add Node allows the user to add a new lest location of the 
Components section MIB. 

Add List permits the addition of a new lest case list to the 
sequence of tests to be run. Anew test case list can be added 
to either a newly added node or to a node which already has 
test cases defined. Of course, the list may also be added by 
clicking on the "Add List" menu choice. 

Clicking on "Load" loads information from the database 
for use by traffic driver/checker 44. 

"Delete" erases any test case or node which has been 
previously marked for deletion. 

"Undelete" allows restoral of a previously-deleted test 
node, in case the node was deleted by mistake. 

The first three functions have the same uses as previously 
discussed under "Components," viz. deleting, loading, and 
undeleting test cases. 

The fourth function, "Duplicate," allows the user to copy 
a previously-defined test case, and make a new test case by 
changing some of its parameters. An example of this feature 
would be performing the same series of tests on a different 
IVR in another city. Duplicating the test case and changing 
the telephone number to be dialed will create a new test case 
without the need for re-typing all the other information. 

The "HELP" files contain sections useful to the user in 
operating the traffic driver/checker 44. 

FIG. 3 shows the administration screen display which 
illustrates the description of other clickable areas. 

1. An item is shown as selected by a color change of the 
first image in the item. 

2. The test cases in a list can be moved by drag and drop, 
by clicking and holding the left mouse button on a test case. 
In the upward direction, the test case inserts before the test 
case it is dropped on. In the downward direction it inserts 
after the test case it is dropped on. 

3. CUcking on the second colored bitmap invokes the test 
case editor for the test case. Because there is only one copy 
of a test case (a row in its test case table), changing the 
values in the test case editor change the values for this test 
case in every tree view leaf. 

4. The OID of an item is show by clicking with the right 
mouse button. 

5. By clicking on the plus or minus bitmap one can 
collapse or expand the tree. 

Test Cases Tree View 

6. Clicking with the left mouse button on the second 
bitmap in a test case tree view leaf invokes the Test Case 
Editor. Because there is only one copy of a test case (a row 
in its test case table) changing the values in the test case 
editor change the values for this test case in every tree view 
leaf. 

7. Clicking with the right mouse button, or click and 
holding the left mouse button on the first test case leaf 
bitmap enables one to drag the test case and drop it into a 
Test List. 
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FIG. 4 shows the database promotion screen display. This 
screen display enables one to change the destination host 
and port at which the trafSc driver/checker 44 is responsive. 
For correct operation, for example, the trafiSc driver/checker 
44 is responsive on port 161 (the standard SNMP port). In 
the preferred embodiment, the traffic driver/checker 44 
cannot run concurrently with other SNMP application in a 
single Windows NT machine. 

The single step box is useful to trickle promotion traffic to 
the trafiSc driver/checker 44. If checked, another button 
labeled Step is made visible, and clicking on this button 
promotes the contents of one traffic driver/checker 44 table 
at a time. 

The test case editor of FIG. 5 shows the parameters for a 
test case. There are two parameter data types, string and 
integer. For string data there are two sub types of string, 
internal which is less than 255 bytes long, and external string 
types that consists of a file and its associated file name. 

The check box next to the string name indicates if the 
string is internal or external. When clicked to check the box, 
a browse menu is prompted to select the file name of the file 
that contains the external string. 

For integer data the valid range defined in the MIB is 
shown. Values outside this range are not allowed. 

The box labeled Test Case Mode sets the mode of the test 
case. Simulation modes are Timeout, Pass and Fail. The Run 
modes are Learn and Test. 

Test Case modes are learn and test. The Learn mode is 
used when running a test case to discover how it reacts to a 
know good component. In learn mode the result should be 
recorded by the test case so that is can determine a pass or 
fail when running in the test mode. The Test mode is used 
to run a test case against a component to determine if the 
component is working. 

One special result is possible when running in the test 
leara, or test mode. This is a fail case, and occurs when the 
test case event in the test case OCX is unassigned (no entry 
address). The traffic driver/checker 44 considers this is a test 
failure, and this result may be attributed to an implementa- 
tion error. 

FIG. 6 shows the Test Executive screen display of the 
traffic driver/checker 44. 

On the left side of the screen display are a number of 
SNMP counters. In normal operation the error counters are 
preferably at zero. 

The community field shows the last community name 
received. If there is no record of a community name, the 
SNMP request is not processed. To add a community name, 
either accept the name in the community field, or type the 
desired name and press the "add" button. Similarly, one can 
remove a name with the delete button. 

The reminder of the fields show the last 01 D in and out, 
and the SNMP data packet. The format of these packets is 
described in the text "Understanding MIBs" identified 
above. 

In addition to the foregoing information, a log file is 
prepared. Errors are recorded. All messages can be recorded 
by selecting the messages button. Log messages are written 
in either a hex version of SNMP packet format, or for errors, 
in plain text. 

FIG. 7 shows the Program Control screen display. This 
screen display shows a Test List with a test result of "Test 
Complete: Timeout". The test case with green bitmaps 
respond with a "pass". The test case with the black bitmap 
timed out, and Program Control has stopped the lest case, 
ITie test cases with the yellow bitmap are "ready to run", but 
cannot run because of the test case timeout. Each test case 
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is a separate windows task that communicates with the 
program manager through a UDP socket. 

One can run a test case list from the Program Control 
Window. This is accomplished by clicking the right mouse 
5 button on the bit map before the Test Case List name. This 
feature is intended to test the Test Cases without having to 
poll the traffic driver/checker 44 run time from an SNMP 
manager. 

A test case can be canceled by night clicking on the test 
10 case icon in a test Case List. This sends a stop message from 
program control to the Test Case. If the Test Case does not 
stop within 30 seconds, Program Control will timeout and 
destroy the Test Case process. 

Test cases are built in the following manner. Before 
building the test cases for a system, a review of the particular 
system is needed. The goal for the review is to provide all 
the information needed to build the test cases. The review is 
preferably a four-step procedure, as follows: 

1) Analyze the services provided to customers. The analy- 
sis should clearly identify what constitutes a working system 
and how the customers determine whether or not the service 
is working. 

2) List the components of the technical systems used in 
providing the service, and what each system must do to 
successfully deliver service to the customer. This creates the 

25 basis for defining a series of pass/fail tests which will be run 
using the traffic driver/checker 44. 

3) For each system, list how it is known whether the 
system is working, firom the customer's viewpoint. This will 
define the normal operation for the system. 

30 4) For each system, determine the conditions which 
characterize whether the system is succeeding or failing in 
its assigned task. These pass/fail conditions will inform the 
traffic driver/checker 44 whether or not the system is ful- 
filling its assigned role in providing service to the customer. 

35 Once the foregoing information has been compiled, build- 
ing actual test cases becomes a three-step process: 

1) Create a MB to define the test process and to tell the 
traffic driver/checker 44 the order in which tests are to 
proceed. 

40 2) Create a series of test cases (test case lists) which 
determine whether the system passes or fails the test. These 
test cases may include any combination of tests defined for 
the systems. 

3) Create a script which runs a single test, for each aspect 
45 of the system which is to be monitored. Collections of these 
test scripts are used to verify that the systems are operating 
properly. 

FIG. 8 illustrates the interrelationship between the MIBs, 

the test case list and the individual test cases. 
50 The following is an example of an MIB. A user of the test 

equipment can add additional MIBs and test cases. 

Each test case (row in the test case table) represents a 

specific test case. Each test case in a lest case table has the 

same syntax parameter list (each parameter is a column in 
55 the lest case table). Each test case in the table may have 

different run time parameters, including the name that the 

program uses to run the test case. 

Each test case table has a unique number, namely a part 

of the MIB OID that identifies the test case table. In the 
60 example below, the unique number is defined by the clause 

{sigModules 7} in the MODULE-IDENTITY section of the 

MIB definition. 
To build a new test case, the names in this section are 

changed to the new test case name. (The text identified 
65 above and entitled "Understanding MIBs" can be utilized to 

obtain a explanation of the ASN syntax required to make this 

change). 
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SIG-USER-TESl'-CASES DEFINITIONS "o BEGIN 
IMPORTS 

MODULE-IDENTITY, OBJECT-TYPE, Integcr32 

FROM SNMPv2-SMI 
TEXTUAL-CONVENTION. DisplayString 

FROM SNMPv2-TC 
MODULE-COMPLIANCE. OBJECT-GROUP 

FROM SNMPv2-C0NF 
sigGeneric, sigModules 

FROM SYSTEMS-INTEGRAnON-GROUP-GLOBAl^REG; 
userTestCases MODULE-IDENTITY 
LAST-UPDATED "9705140000Z" 
ORGANIZATION "Systems [nlcgratioo Group" 

CONTACT-INFO 
'* Duncan Hare 

Postal: Systems Integration Group 
PO Box 3230 
Blaine, WA 98231-3230 
US 

Ul: +1 604 301 0935 
E-mail: cluncan.hare@iname.com 
DESCRIPTION "Initial Version of the simulation 

MIB, version 1.0" 
::- { sigModules 7 } 

RowStatus ::- TEJOTJAL-CONVENTION 

STATUS current 

DESCRIPTION "Row status-destroy is not allowed" 

SYNTAX INTEGER { active(l), 

notInService(2), 
notReady(3), 
createAndGo(4), 
crcateAndWait(5), 
destroy(6) } 

TestCaseMode :;- TEXTUAL-CONVENTION 



STATUS 
DESCRIPTION 



current 

"Mode of test case list. 

If the mode is undefined, the test 



passed, 
result. 
faUed, 
system 

test case 
to the 

in learn mode, 
test case list 
test 

mode to individual, 

individual test cases 

individual test cases." 
SYN-TAX 



skipped. 
If the mode is pass, the test is 

with the result set to expected 

If the mode is fail, the test is 

with result left as set by the 

administrator. 
If mode is set to learn, all the 

runs and set the expected result 

actual result. 

the test cases in the list in run 

If it is set either to test in the 

or, lest or individual for a 

case, the test is ready to run. 
For a TestCasc list, setting the 

causes the TestCaseMode in the 

to define the mode for the 



IN-I-EGER { testUndcfined(l), 
testPass (2), 
testFail (3), 
testljearn(4), 
testltst (5) 

} 

TestCaseStatus ::- TEXTUAI^CONVENTION 
STATUS current 
DESCRIPTION "Test Case state" 
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-continued 



SYNTAX INTEGER { notStartcd(O), 

tcstUnprcparcd(l), 
testReadyToStart(2), 
tcstRuDning(3), 
tcstComplctcSuccssful (4), 
testComplelcTimeout(5), 
U;stCoinpleteFail(6) 

} 

Programlnvocalion ::- TEXTUAL-CONVENTION 
STATUS current 
DESCRIPTION "Program Invocation, complete 

execution path, and start parms" 

SYNTAX OCTET STRING (SIZ:E(0. .255)) 

PhoneNumber ::- TEXTUAL-CONVENTION 
STATUS current 
DESCRIPTION "AT style command string, 

pulse dialing not supported" 
SYNTAX OCTET STRING (SIZE(0. .32)) 

userTcstCascsConf OBJECT IDENTIRER 

::= { uscrTestCases 2 } 
userTcstCascsGroups OBJECT IDENTIFIER 

::» ( uscrTestCascsConf 1 } 
userlbstCasesCompls OBJECT IDENTIFIER 

::- ( userTestCasesConf 2 } 
userTfestCasesObjs OBJECT IDENTIFIER 

::- ( sigGeneric 34 } 
userTestCasesEvents OBJECT IDENTIHER 

::- ( userTestCases 3 } 
userCasesEventsV2 OBJECT IDENTIFIER 

::- { userTestCasesEvents 0 } 

uscrTcstCaseTabl e 



user'ftstCaseEntry 



OBJECT-TYPE 

SYNTAX 

MAX-ACCESS 

STAIXIS 

DESCRirnON 



SEQUENCE OF Use rTestCase Entry 

not-accessible 

current 

"Ttst Case Thblc" 



. { uscrTestCasesObjs 1} 



OBJECT-TYPE 

SYNTAX 

MAX-ACCESS 

STATUS 

DESCRIPTION 



UserlbstCaseEntry 

not-accessible 

current 

"A row in the simulator Test 



Row are ceated when test 



cases are added/ 



UserTestCaseEntry 



Programlnvocalion, 



number 



Test Case" 



program Name 



INDEX { number } 

{ uscrTestCaseT^ble 1 } 
::=. SEQUENCE { 



number 


Integer32, 


name 


DisplayString, 


programName 




status 


TestCaseStatus, 


mode 


TestOiseModc, 


rowStatus 


RowStatus, 


setup 


DisplayString, 


testCase 


DisplayString, 


dN 


PhoneNumber, 


timeout 


Integer32 


}□ 





Integer32(0. .32767) 
read-create 
current 

"The Test Case Lists Number" 



OBJECT-TYPE 
SYNTAX 
MAX-ACCESS 
STATUS 
DESCRIPTION 

{ UserTestCaseEntry 2 } 
name 

OBJECT-TYPE 

SYNTAX DisplayString(SIZE(64)) 

MAX-ACCESS read-create 

STATUS current 

DESCRIPTION "Name of the Tfest Case Ust 

:> { UserTestCaseEntry 3 } 



OBJECT-TYPE 
SYNTAX 



Programlnvocalion 
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-continued 





MAX-ACCESS rcad-CTcatc 




SIXIUS current 




DESCRIPTION "Name of the Tc&i Case List 


Test Case" 


::- { uscrTestCascEntry 4 } 


status 


OBJECTTYPE 

SYNTAX TfestCascStatus 
MAX-ACCESS read-create 
STATUS current 
DESCRIPTION "Status of test case." 
:> { userTestCaseEntry 5 { 


mode 


OBJECT-TYPE 

SYNTAX TestCaseMode 
MAX-ACCESS read-create 
STATUS current 
DESCRIPTION "Mode of test case." 
::» { userTestCaseEntry 6 } 


rowStatus 


OBJECT-TYPE 

SYNTAX RowStatus 

MAX-ACCESS read-create 

STATUS current 

DESCRIPTION "Status of Test Case Lists 


entry row" 


:> { userTestCaseEntry 7 } 
This seaion of the MIB Is Test Case dependenL 


setup 


OBJECT-TYPE 

SYNTAX DisplayString 
MAX-ACCESS read-create 
STATUS current 
DESCRIPTION "IVR Response to call" 
::o { userTestCaseEntry 8 } 


dN 


OBJECT-TYPE 

SYNTAX PhoncNumber 
MAX-ACCESS read-create 
STATUS current 
DESCRIPTION "Phone Number to Call in 


test" 


::- { userTestCaseEntry 9 } 


testCase 


SYNTAX DisplayString 
MAX-ACCESS read-create 
STATUS current 
DESCRIPTION "Phone Number to Call in 


test" 


::= b5 UserTestCaseEntry 10 } 


timeout 


OBJECT-TYPE 

SYNTAX tnteger32 

MAX-ACCESS read-create 

STATUS current 

DESCRIPTION "Phone Number to Call in 


test" 


::= { userTestCaseEntry 13 } 


This section of the MIB Is Preferably Required for all test 



cases. 



uscrTestCaseGroup OBJECT-GROUP 

OBJECT { number, 

name, 

programName, 
mode, 
status, 
rowStatus, 
setup, 
testCase, 
dN, 
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-continued 



Driver/Checker" 
userlts tCaseCompliance 

timeout values" 



END 

□ 



timeout 

STAI^US current 
DESCRIFHON "Objects in Traffic 

::« { uscrTcstCascsGroups 1 { 

MODULE-COMPUANCE 
STATUS current 
DESCRIPTION "All test cases must have 

MODULE useiTestCases 
::- { userTestCasesCompls 1 } 



The test case MIB is added to the compliance section of 
the MIBs included, and to the capabilities part of the MIB 
having the proper file. In addition the new MIB name is 
added to the include file (sys-mib.inc ). 

The MIB is then complied with the m2.bat file provided 
in the MIB subdirectory. At a DOS prompt enter the com- 
mand m2 (which contains the MIB compiler command line) 
is entered. When the MIB complies correctly, the output of 
the MIB compiler (sys-mib.imQ is loaded into the Database 
Load menu option. 

Tests of systems can be carried over into multiple levels. 
System-wide service assurance tests can be built using two 
different methods: centralized and distributed testing. Each 
method has its advantages and disadvantages. 

In the centralized model, every test is run from a single 
system, the centralized test unit tests every other node within 
the system. This method makes test control easy as each test 
is scheduled from a single subsystem; it is likely the best 
method for small system configurations, where all the com- 
ponents are in a single building, or on a single LAN 
segment. This configuration may be sufificient for networked 
systems with two or three interconnected LANs, 

The disadvantage of the foregoing test implementation 
arises from the nature of networked systems. If a link 
between the test system and other networked computer 
systems fails, it is difficult to test whether or not the other 
LAN segments are functioning proper, because the target 
system cannot be reached. The link between the test system 
and the other subnetworks must be repaired before useful 
testing can continue. TTiis disadvantage can be reduced 
significantly if more than one link between the test system 
and the computer subnets can be made available. 

Distributed testing implementations require a test system 
in each major area of the network. A business with several 



distributed oflices may want to be able to test within each 
major city, and aggregate the reports to a centralized office 
for trouble ticket handling. Subnets can be tested locally, 
reducing the likelihood of link failure between the test 
system and the applications to be tested. Redundancy is also 
provided, because more than one test unit is resident on the 
network. In the unlikely event of a test system failure, 
another system can resume testing for the failed unit. 

Each individual application on each system node is tested 
on a pass/fail basis. When the application fails, the test 
apparatus of the invention performs diagnostic tests on the 
node to determine the location of the failure, and whether the 
fault is likely to be hardware or software related. Each node 
which is involved in providing customer service is fault 
tested, on a repeating basis to ensure that customer service 
is being provided in a satisfactory manner. 

The test equipment provides node-by-node diagnostic 
tests when required, to analyze a reported test failure. The 
node-by-node testing is performed under command of the 
enterprise automation system. 

MIBs are addressed by way of Object Identifiers (OlDs), 

An example of an OID is: 1.3.6.L4.L2392. 2. 5, 2. 2. L 
The prefix, L3.6. 1.4.1 refers to an SNMP MIB. The number 
2392 is the test apparatus MIB enterprise number The heart 
of the MIB numbering scheme is: 2. 5. 2. 2. 1. 

The following Table 9 illustrates the OID table number- 
ing. In general OIDs represent a static committed interface 
between MIBs users. Within the static definition of elements 
addressed by OIDs are dynamic tables. Th& dynamic tables 
have rows that are implementation, both for internal use by 
the test apparatus and the test case Components and Test 
Case Lists, defined above. The test apparatus makes exten- 
sive iise of dynamic tables. 



TABLE 9 



OID Name 



1.3.6.3.4.1.2392.2.3.2.1 Port Card List the cards used in a Service Meter System for 
l^ble telephone calling out or calling in. 

1.3.6.1.4.1.2392.2.3.2.2 Port Contains the ports (telephone lines) on a card 
Uble 

1.3.6.1.4.3.2392.2.5.2.1 Node Names of the Components in the system that arc to be 
Thble tested. The rows in the node table contain the node 

names. 

The OID of the first row is: 

1.3.6.1.4.1.2392.2.5.2.1.3 of the second: 

J. 3.6.1.4. 1.2392.2.5.2.1 .2 and so on sequentially. 

1.3.6.1.4.3.2392.2.5.2.2 Tbst Case Table to contain the test case lists. Each list has a 
lists name and an object identifier. 

Uible Polling (doing an SNMP GET Request) to the OID 
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TABLE 9-continued 



OID 


Name 


Use 






1.3.6.1.4.1.2392.2.5.2.2.X.4 the Test Case List status 






element. 


1.3.6. 1.4.]. 2392.2.5 .2.3 


Test Case 


This contain the run time information (pass, fail and 




List T^ble 


test result) about the test case. 


1.3.6.1.4.1.2392.2.99 


Test Case 


Enables the SNMP manager to restart and stop the 




Manager 


Service Meter executive. This is the ONLY variable 




we recommend the SNMP manager sets (write to) in 
the service meter MIB. 


1,3.6.1.4.1.2392.3.31.1 


Voice 


Contains the Voice Call test cases. May be modified 




Call Test 


to add additional test case parameters. 




Case 






Table 




1.3.6.1.4.1.2392.3.34.1 


User Test 


Contains User Test cases, for test cases where a 




Case 


Unique MIB is not required. 




Tabic 




1.3.6.1.4.1.2392.3.37.1 


Voice 


Contains the Voice Call Answer test cases. Can be 




Answer 


used to track call arrivals at call center agents. May 




Test Case 


be modified to add additional test case parameters. 




T^ble 




1.3.6.1.4.1.2392.3.40.1 


Data Call 


Contains the Data call answer test cases. Can be used 




Answer 


to track screen pops at call center agents. May be 




Test Case 


modified to add additional test case parameters. 




Ibble 




1.3.6.1.4.1.2392.3.43.1 


Data 


Contains the Data Transaction test cases. Can be 




Transacti 


used to test transactions into legacy systems. May be 




on Tfesi 


modified to add additional test case parameters. 




Case 






Table 




1.3.6.1.4.1.2392.3.46.1 


Ivr Call 


Contains the IVR Call flow test cases. Can be used 




Flow Test 


with voice recognition to test IVR prompts. May be 




Case 


modified to add additional test case parameters. 




Table 




1.3.6.1.4.1.2392.3.49.1 


Ivr Port 


Contains the IVR Port ftow test cases. Can be used to 




Test Case 


monitor the state and statistics of a set of IVR ports. 




Table 


May be modified to add additional test case 



When the test apparatus of the invention first runs, it 
accesses the MIB, and performs an initialization routine 
which "walks" through the MIB and creates all the test cases 
necessary [as defined in the MIB]. The test apparatus does 
not run the test cases until commanded to do so; however, it 
is ready to run the tests due to the setup/initialization routine 
it performs. This process is referred to as MIB discovery. 

As noted above, MIB has dynamic tables contained 
therein. The Object Ids of these tables row change as 
Components or Test Case Lists are deleted and the database 
expunged. When the database is expunged, and promoted 
from the Administration to the Executive routine, each item 
in the MIB must be rediscovered. 

In addition, the OIDs of Components is dependent on 
their position in the test case list table. According to the 
preferred embodiment, new components and test lists are 
added at the end of the table. 

Yho Component and Test Case List tables should be 
carefully prepared and built if automation is being utilized. 
The Component, Test Case List and Test Case run time OIDs 
are the way that automation package understands what an 
individual test case list is testing. These OIDs will change if 
there is a deletions in the MIB tables and the MIB database 
is expunged. 

The functions of the test apparatus is operated under 
control of an enterprise automation package, such as HP 
Open View 40. The enterprise automation system commands 
the test apparatus to perform one or more tests on a repeating 
basis. These commands, which cause the MIB to run a test 
case and provide a result to the automation system, are 
considered to "poll the MIB." The trafiSc driver/checker 44 
advises the automation system whether the target device 
passed or failed the test. 



The SNMP Manager poUs (issues and SNMP GET 
request) the Test Case List state element whose OID is: 
1.3,6. 1,4.1. 2392.2.5.2.2.X.4, where x is the number of the 
Test Case List, and ranges from 1 to 999. ITie upper limit of 
40 999 is a traffic driver/checker 44 restriction on the maximum 
number of rows in a table. 

Polling the Test Case list result can return any one of 
seven states. When the SNMP manager reads the state, two 
45 actions are taken. The first is to return the state. The second 
is to advance the Test Case List to its next state. 

The Test Case Lists contains a memory for diagnosing 
failed tests. When a test fails, the failing tests* run time OID 
is placed in the Test List at OID 1.3.6.1.4.1,2392,2.5.2.3.x.6, 
where x is the number of the Test Case List. 

The table row referenced by this OID contains informa- 
tion from the failing test (if the test is well behaved) at the 
OID "failed_test_case.ll. This table element is a 255 byte 
55 character string. The contents of this string are test case 
dependent, and are mostly defined by the test case designer. 

There are three test failure results generated internally by 
the traffic driver/checker 44. These are; 

gQ 1. Test Fail: Timeout. 

2. Test Fail: Simulator. 

3. Test Fail: No test program assigned. 

The states, the color of the state in the Program Control 
65 tree view, an explanation of the states meaning and the next 
action taken when the state is read is contained in the table. 
Table 10, below. 
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TABLE 10 
Progtam Control 



State 


Bitmap Color 


Next ActionD 


Not StartcdD 


Black 


Unprepared □ 

All test case programs are started. 
They should return a "start" 
message to program control. 
When all have started then the 
state will change. 


Unprepared 


Blue 


Trying to StartD 
As soon as the routine to prepare 
the test cases begins, the state is 
set to "Trying to start". 


Tr>ing to start 


Grey 


Ready to StartD 

Parameters are sent to the test 

cases. When all pramieter are 

sent, a "setup" message is sent to 

each test case. When all test 

cases respond to the setup 

message, the state changes to 

Keaoy lO oiari.LJ 

This state transition is used to 

enable the test case to initialize 

themselves 

There is no timeout period for a 
test case to initialize 


Ready To Start 


Yellow 


Running. □ 

As soon as the test case list (all 
test cases are Ready to Start) the 
first test case is ruin. 


Running 


Cyan 


Pass, timeout or Fail. □ 

The state of a test case when it is 

testing a component 


Pass 


Green 


Unprepared 


Timeout 


Red 


Not Started (the test case is 
stopped) 


FaU 


Magenta 


Unprepared 



It is possible to change (write to) the MIB with the SNMP 
management system, with the exception of the OID 
1 .3.6.1.4.1,2392.2.99, which is used to restart the Executive. 

Automation packages are rule -based systems; that is, they 
run tests and use a specified rule to determine whether the 
response passes or fails to meet the criteria. The rules 
embody the requirements passing the test, and identify the 
meaning of the result received. 

An example of a rule affecting the interpretation of test 
results is "The ABC system will be out of service for 
maintenance every day between midnight and 2:00 a.m." 
Clearly, any test which runs on this system during the period 
from midnight to 2:00 a.m., will fail every time the test is 
run. It may be decided that the way to implement this rule 
is not to run tests on applications dependent on the ABC 
system during the hours of ABC*s scheduled downtime. 
Thus, in this case, the enterprise automation package may be 
defined so tests are run only during the remaining 22 hours 
of each day. 

When a service assurance lest fails a predefined number 
of times, the traflic driver/checker 44 uses information 
obtained from the test results to run a series of diagnostic 
tests that determine which subsystem failed. The informa- 
tion is passed back to the enterprise automation package. 
The enterprise automation package can then use the infor- 
mation to initiate a trouble ticket and route the ticket to the 
correct technical group for repair. 

The following \^sual Basic program is an example of a 
simple test case. The six events that must be consider are: 
Form Load, Form Terminate, OnEndTest, OnSel Farms, 
OnSetup and OnRunTest. 

In the example shown. Form Load sets the Test Case 
Caption, 'llie port number issues to distinguish one test case 
from another. The port number is in the 3000 range. The port 
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number is built from adding 30(X) to the unique test case 
value in the Test Case List MIB table (OID 
1.3.6.1.4.1.2392.2.5.2.3,x), where x is the unique row value 
of the test case in the Test Case List table. 
5 OnEndTest is a message sent from Program Control to the 
Test case. Both this event and Form Terminate mean end, 
free all of the resources and end. If the test case does not end 
after receiving an OnEndTest event. Program Control waits 
for a test complete event from the test case for 30 seconds. 
If this is not received. Program Control cancels the test case 
process. 

The OnSetParms event occurs the same number of times 
there is a test case parameter in the MIB. In the preferred 

j5 embodiment, there are a maximum of 16 parameters allowed 
for a Test Case (a Test Case Editor limitation). In the 
following sample the parameter strings are stored in a string 
array. MIB values with sufiSxes 8 to 16 arc sent to a test case. 
Other MIB parameters (1 to 7) are used by the Program 

20 Control. 

OID: is the unique test case number in the test case list. 

Farm: is the parameter value, always a string. 

ParmType: Internal for up to a 255 byte string. External is 
25 a file, whose file name is contained in Farm. 

The OnSetupTest event occurs twice. This event occurs 
First time when all parameters have been sent to the test 
case. At this point the test case should set up the run time 
environment. 

30 A second time of the event occurs with an internal string 
that contains information from previously successful test 
cases. This string is used to pass information between test 
cases in the Test Case list. It contains concatenated infor- 
mation from the: 

TestDataType, TestData and TestResult 

OnRunTest: Run the test case. There are no parameters. 

OCX properties. 

There are three test case properties, as discussed above. 
TestDataType: 

Interna l_text only. 
TestData: 

A short string that contains information to be passed to 
subsequent test cases. The aggregate length of this 
string from all test cases in a test list is 255 bytes 
TestResult 

True if the test passed. False if the test failed. 
The Program Control of the trafiBc driver/checker 44 can 
50 run and terminate a test case without the use of an SNMP 
Manager. The right mouse button click on the test list bitmap 
in the Program Control windows runs the test list. ITie right 
button clicked on the test case bitmap stops the test case. 
The Test Case OCX provides a Writelog method. This, in 
55 conjunction with the logLevel property controls the OCX 
bases test case log. The OCX can log all messages between 
Program Control and the Test Case. The logLevel is set to all 
to collect all messages. 

The logCounl property controls how many messages are 
60 written into a log file. The logCount value default is 4096. 
For example: 
LogLevel=all 

Call writeLog( info, "This is an information message") 

65 Or 

LogLevel«»error 

Call writeLog( error, "This is an error message") 
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The following is an exannple of the Visual Basic Test case. 



Dim instring As String 

Dim tcParms(8 To 15) As String 

Private Sub Form_Load( ) 

instring = CStr(smTX:X 3. Local Port) 

For ml. Caption = Forml. Caption + ":" + instring 

End Sub 

Public Sub Form_Terminate( ) 

End 
End Sub 

Private Sub smTCXl_OnEndTest(ByVal action As 
smTestCaseXControl.TxClose Action) 
End 
End Sub 

Private Sub.smTCXl_OnSetParms(ByVaI oid As Long, 
By\^l parm 

As String, ByVal parmTypc As smTestCascXControl.TxtcResult'IVpe) 
tcparms(oid) = parm 
End Sub 

Private Sub smTCXl_OnSetupTcstCBy\b! data As String, ByVal 
datalVpe As smTestCaseXControl.TxtcRcsultlVpc) 
'(* setup test case here-all parms arc in *) 
End Sub 

Private Sub smTCXl_OnRunTest( ) 
'mn Tfest 

smTCXlTestDataiype - internalText 
smTCXl.TestData - Forml. Caption 
smTCXl.TestResult - True 
End Sub 



In summary, the Management Information Base (MIB) is 
the basic building block for Enterprise Management, The 
MIB identifies the component applications in the system, 
associates each component with one or more test cases, and 
directs the flow of tests over the network or subnet to be 
tested. The MIBs are used to control the flow of testing, as 
well as to pass information from the SNMP agent back to the 
traffic driver/checker 44. 

The MIB structure uses databases and lists; some lists are 
used as indexes; others are cross-references between the 
MIB and the "human names." An example is the name 
"Test CaseListG roup" — an ASCII string which is more 
meaningful to humans than "1.3.6.1.4.1.2392. 3. 4. 1. 3",the 
Object ID belonging to TestCaseListGroup. 

While the preferred embodiment of the invention has been 
disclosed with reference to a specific test apparatus and 
methods of operation thereof, it is to be understood that 
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many changes in detail may be made as a matter of engi- 
neering or software choices, without departing from the 
spirit and scope of the invention, as defined by the appended 
claims. 
5 What is claimed is: 

1. A method of testing transaction systems, comprising the 
steps of: 

coupling test inputs to the transaction system, at least one 
10 said test input being of an analog type as input by a user 
of the transaction system; 

said coupling step including coupling a plurality of dif- 
ferent output probes to respective sub-systems of the 
transaction system; 

receiving from one or more points of said transaction 
system respective responses to the test inputs, one 
response being an analog response; 

coupling a plurality of response check points via a cor- 
20 responding plurality of response lines to receive said 
responses; 

utilizing an MIB structure to carry out the coupling and 
receiving steps; and 
2^ utilizing enterprise automation equipment to control the 
MIB structure and identify a fault in the transaction 
system based on the responses received therefrom. 

2. The method of claim 1, further including periodically 
monitoring the traasaction system at periodic intervals dur- 

30 ing regular operation of the transaction system to verify 
correct operation thereof. 

3. The method of claim 1, further including dialing a 
telephone number of the transaction system, and receiving a 
voice response that assures correct operation. 

35 4. The method of claim 1, further including using a tester 
having response inputs and test signal outputs, and further 
including coupling commands between the enterprise auto- 
mation equipment and said tester using an Internet connec- 
tion. 

40 5. The method of claim 1, further including using bound- 
ary probes coupled to the transaction system to determine 
which sub-system of the transaction system has failed. 

♦ ♦ * ♦ * 



09/30/2003, EAST version: 1.04,0000 



